Image enhancement methods and apparatus therefor

ABSTRACT

The invention relates to methods, devices and computer programs for processing images. The dynamic range of an image is adjusted to allow more detail in the original image to be reproducible, whilst preserving the natural look of the image. Improvements to lossy data compression/decompression algorithms are proposed which include elements of spatial non-uniformity, for example modifying relatively dark portions of an image to a greater extent than relatively bright portions of the image. The algorithms may be made reversible by storing parameters describing the compression process with a compressed image, which can then be used to reverse the process when decompressing the image.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application relates to, claims priority on, and is anational application filed under 35 U.S.C. § 371 of prior internationalpatent application PCT/GB02/01675 filed Apr. 10, 2002, which is herebyincorporated by reference in its entirety.

FIELD OF THE INVENTION

The invention relates generally to image processing, and moreparticularly to image improvement. The invention may be used for dynamicrange compression and colour constancy for images both in digital andanalogue form. In one aspect, the invention relates to image datacompression.

BACKGROUND OF THE INVENTION

Dynamic range is the ratio between intensities of the brightest anddarkest recordable parts of an image or scene. A scene that ranges frombright sunlight to deep shadows is said to have a high dynamic range,while indoor scenes with less contrast will have a low dynamic range.Note that depending on the scene contrast, it may or may not be possibleto capture the entire range with a digital camera. In recording sceneswith very high dynamic range, digital cameras will make compromises thatallow the capture of only the part of the scene that is most important.This compromise is needed because no camera or output device of any kind(including the human eye) can reproduce the nearly infinite dynamicrange that exists in real life.

In an input image which contains a level of background noise, the noiselevel defines the darkest part of the image for the purpose of thedynamic range. Amplifying the brightness of the darkest parts of animage containing noise tends to decrease the dynamic range, as thedarkest meaningful parts of the image, defined by the noise level, areincreased in brightness.

Electronic cameras, based on CCD detector arrays, are capable ofacquiring image data across a wide dynamic range on the order to 2500:1.This range is suitable for handling most illumination variations withinscenes (such as high contrast scenes taken on a sunny day). Typicallythough, this dynamic range is lost when the image is digitised,compressed by a lossy compression algorithm (like JPEG or MPEG) or whenthe much narrower dynamic ranges of print and display media areencountered. For example, most images are digitised to 8-bit/colour band(256 grey levels/colour band) and most displays and print media are evenmore limited to a 50:1 dynamic range.

It would be desirable to try to find some transform which can compressthe dynamic range of an image to allow more detail in the original image(which may be colour or non-colour, still or video) to be reproducible,while preferably preserving the natural look of the transformed image.Let us call such a transform an “image improvement transform”.

Since it is important to be able to improve images (video) in real timethe computational efficiency is an important parameter of imageimprovement systems. To provide low power consumption, smallness in sizeand the low cost of an integrated circuit (IC) or other deviceimplementing an image improvement algorithm, it would be desirable toprovide an analogue implementation of an image improvement algorithm.Another advantage of an analogue implementation of image improvementalgorithm is that would allow the avoidance of the addition ofdigitising noise caused by analogue to digital conversion (ADC).

It would also be desirable to provide an improvement to lossy datacompression algorithms such as JPEG, to allow a greater amount ofmanipulation of compressed/decompressed images.

Let us consider the current popular image enhancement algorithms.

Homomorphic filtering is an image enhancement technique which uses thefollowing algorithm:I _(out)=exp[HPF _(Ω)(log(I))]=exp[log(I)−LPF _(Ω)(log(I))],

where HPF is a high-pass 2D spatial filter function.

U.S. Pat. No. 5,991,456 describes an image enhancement algorithm systemnamed Retinex™. This uses the following algorithm:

${I_{out} = {\sum\limits_{i = 0}^{N}{W_{i}\left( {{\log(I)} - {\log\left( {{LPF}_{\Omega_{i}}(I)} \right)}} \right)}}},$

where W_(i) are constants.

A disadvantage of this algorithm is that it is incapable of optimisationof the high-frequency (spatial) components

$\left( {{with}\mspace{14mu}{frequencies}\mspace{14mu}{higher}\mspace{14mu}{than}\mspace{14mu}{\max\limits_{{i = 1},\ldots\;,N}\left( \Omega_{i} \right)}} \right)$of an image, thus sharp transitions between dark and bright regions ofan image will exist on the transformed image. Thus a disadvantage of theRetinex algorithm is the excessive contrast of small high-contrastdetails of images. A further disadvantage is its computing powerrequirements. Typically, image processing takes hours on a conventionalcomputer workstation. Therefore, Retinex is ill-suited to most practicalapplications

Techniques known as histogram modification (or equalisation) algorithmsare also known. There are many histogram modification algorithms. Suchalgorithms modify the transfer function H(..) of the whole image:I _(out)(x, y)=H(I(x, y)).

More particularly, histogram equalisation (cumulative histogram-based)as well as other histogram modification-based algorithms may bedescribed using the following equation:

$\begin{matrix}{I_{out} = {\frac{1}{M}{{CumHistogram}(I)}}} \\{= {\sum\limits_{i = 0}^{N}{{\theta\left( {I - I_{i}} \right)}\left\langle {\delta\left( {I - I_{i}} \right)} \right\rangle}}} \\{{\equiv {\sum\limits_{i = 0}^{N}{\theta\left( {I - I_{i}} \right)\frac{1}{M}{\sum\limits_{k,l}{\delta\left( {{I\left( {k,l} \right)} - {I_{i}\left( {k,l} \right)}} \right)}}}}},}\end{matrix}$

where M is the number of pixels in the image I, I_(i) are the levels ofgrey (e.g. in case of grayscale images N=255, I₀=0, I₁=1, . . . ,I₂₅₅=255),

${{\delta(x)} = \begin{matrix}1 & {{{if}\mspace{14mu} x} = 0} \\0 & {{{if}\mspace{14mu} x} \neq 0}\end{matrix}},{{\theta(x)} = \begin{matrix}1 & {{{if}\mspace{14mu} x} \geq 0} \\0 & {{{if}\mspace{14mu} x} < 0}\end{matrix}},$and (k,l) are the pixel coordinates. The averaging operation

..

could be considered as the low-pass filtering LPF_(Ω) with the cut-offfrequency

${\Omega = 0},{{{so}\mspace{14mu} I_{out}} = {\sum\limits_{i = 0}^{N}{{\theta\left( {I - I_{i}} \right)}{{{LPF}_{0}\left( {\delta\left( {I - I_{i}} \right)} \right)}.}}}}$

Histogram equalisation algorithms can be limited in utility in caseswhen the histogram of the whole image is uniform, whereas some parts ofthe image are too bright and the others are too dark. In such casesdifferent transfer functions should be applied to different regions ofimages.

Local histogram equalising (also referred to as adaptive histogramequalising) techniques are also known. The techniques are adaptive inthe sense that in different regions of a picture different transformsare applied to equalise the histogram of each particular region. Thereare many implementations, which can be represented by the formula:

${I_{out} = {\sum\limits_{i = 0}^{N}{{\theta\left( {I - I_{i}} \right)}{{LPF}_{\Omega}\left( {\delta\left( {I - I_{i}} \right)} \right)}}}},$

where Ω is the spatial frequency related to the size of regions withequalised histogram (if the size of “window” is r, the size of an imageis L, than

$\left. {\Omega = \frac{L}{r}} \right).$

However, known local histogram equalisation (adaptive histogramequalisation) techniques are highly computationally intensive anddifficult to embody in real time applications. Furthermore, in manycases a rectangular window around each pixel is used for histogramequalisation in this window, thus such transforms cannot be consideredas spatially smooth. Such non-smoothness of a transform could increasethe noise and artefacts of transformed images.

It is an object of one or more aspects of the present invention toprovide a method of improving an image. Preferably, the image may becreated with digital or analogue data. Preferably, the method should besuitable for application to both colour and non-colour images.

Another object of one or more aspects of the present invention is toprovide a method of improving images in terms of dynamic rangecompression, colour independence from the spectral distribution of thescene illuminant, and colour/lightness rendition.

Another object of one or more aspects of the invention is to provideanalogue implementations of the suggested image improvement algorithm.Such implementations will generally be small sizes and have low powerconsumption, which are desirable characteristics in many image capturedevices.

Still another object of one or more aspects of the invention is toimprove lossy image compression procedures, such that aftercompression-decompression images can retain a relatively wide dynamicrange, which is important if certain types of post-processing are to beperformed, for example in medical image processing techniques (such asthose used in X-ray and Magnetic Resonance Imaging (MRI scans) or in themanipulation of photographic images using image editing software.

In accordance with one aspect of the present invention, there isprovided an image processing method comprising the step of processing aninput signal to generate an adjusted output signal, wherein theintensity values I(x,y) for different positions (x,y) of an image areadjusted to generate an adjusted intensity value I′(x,y) in accordancewith:I _(out)=Σ_(i=0) ^(N)α_(i)(I)LPF _(Ω) _(i) [P _(i)(F(I))]·Q _(i)(F(I))·Q_(i)(F(I))+β(I),

where P_(i)(γ) is an orthogonal basis of functions of γ defined in therange 0<γ<1; and Q_(i)(..) are antiderivatives of P_(i)(..):Q_(i)(F(I))=∫₀ ^(F(I))P_(i)(η)dη or an approximation thereto,LPF_(Ω)[..] is an operator of low-pass spatial filtering; a Ω_(i) is acut-off frequency of the low-pass filter, and F(..) is a weightingfunction.

The family of possible transforms, which could be represented by theabove algorithm, are referred to herein as the Orthogonal Retino-MorphicImage Transform (ORMIT).

The functions Q_(i)(..) may be approximations to the antiderivatives ofP_(i)(..) within a range of 25%, more preferably 10%, of variation fromthe antiderivatives. Ω is preferably non-zero. F(..) preferably varieswith I in an asymmetrical manner, β(I) is preferably not equal to zero.Ω_(i) is preferably different for different i such that thetransformation has a different degree of spatial nonuniformity indifferent brightness domains.

The image improvement algorithm is preferably asymmetric in thebrightness domain, such that the algorithm has a greater effect on darkregions of an image than relatively bright regions. The algorithmpreferably performs dynamic range compression so as to displacerelatively dark portions of the image within the range of intensityvalues to a greater extent than relatively bright portions, therebyachieving this asymmetry. The image contrast in the dark regions ispreferably also increased to a greater extent than that in the lightregions. The effect of this is to mimic the behaviour of the human eye,which has a relatively high dynamic range extending well into the darkregions of an original image, by reproducing these dark regions in alighter region of the reproduced image. Furthermore, when used,reversibly, in combination with a standard lossy compression technique,as will be described in further detail below, the dynamic rangecompression functions to increase the amount of useful data retained inthe dark regions of an image. Since the human eye has a very highdynamic range (apprx. 10⁵:1) whereas the dynamic range of the opticnerve is about 100:1, it is clear that the eye performs a transformwhich strongly compresses the dynamic range.

The algorithm according to this aspect of the invention preferablyincludes an element of spatial nonuniformity in the transfer functionwhich is applied to the image. That is to say, the transfer function mayvary across the image, as in the case of local histogram equalisation.

Furthermore, the shape of the transfer function applied is preferablynonlinearly adaptive in the spatial domain. That is to say, the transferfunctions which are applied in different parts of the image need bear norelation to one another in shape, to improve the handling of contrastpresent in an original image. The transform preferably makes use oforthogonal functions, thereby to improve computational efficiency. Theorthogonal functions are in one embodiment Legendre polynomials, inanother piecewise linear mapping functions. These transforms arepreferably carried out in the analogue domain.

The algorithm is preferably smooth both in the brightness domain and thespatial domain.

A further preferred characteristic of the algorithm is reversibility.

According to a further aspect of the invention there is provided amethod of compressing an image signal, comprising conducting areversible dynamic range compression algorithm to transform the imagesignal to an enhanced image signal, and conducting a lossy compressionalgorithm to compress the enhanced image signal to generate compressedimage data.

The method preferably comprises generating parameters relating to thedynamic range compression algorithm, and storing same in combinationwith the compressed image data.

According to a yet further aspect of the invention there is provided amethod of decompressing compressed image data, comprising conducting adecompression corresponding to a lossy compression algorithm, andconducting a reverse dynamic range compression algorithm to provide adecompressed image signal.

In one embodiment, a dynamic range compression algorithm is usedreversibly in combination with a standard lossy compression technique(such as JPEG), with parameters describing the image improvementtransform accompanying the compressed image data to enable the reversetransform when decompressing the image. In this sense, the algorithm isto be considered “reversible” if the amount of data describing thereverse transform is not significantly greater than the amount ofstandard image compression data. Preferably, the amount of transformdata is significantly less than, i.e. less than half the amount of, thestandard image compression data. In one embodiment, the transform datais inserted in a header part of the compressed image file.

In a preferred embodiment, the intensity value I(x,y) for each position(x,y) of a greyscale image is adjusted to generate an intensity valueI_(out)(x,y) (improved image) for each position in accordance withI _(out)=α·Σ_(i=0) ^(N) LPF _(Ω) [P _(i)(F(I))]·Q _(i)(F(I))+(1−α)I

Here α is the “strength” of transform, which is preferably, in the caseof image enhancement between 0 and 1. Typically, α may take a value inthe region of 0.5. The nonuniformity parameters Ω_(i) may take differentvalues for different i to achieve best enhancement while retainingnatural appearance. It may be convenient to set all to a constant valueΩ as above.

The proper selection of α (which is the strength of transform), spatialnonuniformity of the transform (the “window” of the transform)represented by Ω, the weighting function F(..) and basic functionsP_(i)(..) provides good quality image improvement, e.g. dynamic rangecompression.

The image is initially represented by electronic format (either inanalogue or in digital form). The proposed algorithm works both withcolour and with grayscale images. In case of colour images the proposedalgorithm could modify either the intensity of image (whereas the hueand saturation are unchanged) or individual colour channels of an image(thus providing the colour correction). Different approaches tocolour-correction of images are also suggested.

The procedure of the reversed transform is suggested. This procedureallows restoring of the original image from the transformed one.

A novel procedure for image compression is suggested. The procedureconsists of an image transform, compressing the dynamic range of animage, as the first stage and a standard lossy image compressionalgorithm, such as JPEG or JPEG2000, as the second stage. Afterdecompression and reversed ORMIT transform the quality of dark regionsof an image is much better than the quality of such regions after JPEGcompression-decompression (with the same size of compressed file). Thusthe dynamic range of images, compressed by the suggested procedure, isincreased.

Schematics, implementing different approaches to analogue hardwareimplementation of the image improvement system, are suggested. The firstschematic is based on a polynomial approach, whereas the second is basedon a piecewise linear mapping function approach. Also suggested is asimplified version of ORMIT, wherein the spatial nonuniformity parameterΩ is set to zero, which has a simpler implementation in analoguecircuitry.

In this case, the ORMIT formula reduces to:

${I^{\prime} = {\sum\limits_{i = 0}^{\infty}{\left\langle {P_{i}(I)} \right\rangle{\int_{0}^{I}{{P_{i}(\eta)}\ {\mathbb{d}\eta}}}}}},$where I(t) is the input (video) image signal, I′(t) is the outputsignal, and where P_(i)(x) defines a basis of orthogonal functions andthe operator

..

means the arithmetic mean (or mathematical expectation) value for theimage. Versions are well suited to the enhancement of video images inreal time.

A motivation for performing image enhancement in analogue circuitryimmediately after capturing a video signal from an image sensor (e.g.CCD) is the fact that it provides potentially the widest dynamic rangeof the output image of the whole image processing system (e.g. the videocapture system, image transmission or recording system and the videooutput system). It results in the visibility of details of an imagewhich would otherwise have been below the level of noise caused by thetransmission, recording and reproduction (in the case of analogue videocameras), or below the noise of digitizing by an analogue-digitalconverter (ADC) and lossy compression (like MPEG).

For real-time applications, the image of a subsequent video-frame ispreferably processed on the basis of information from a previous frame,preferably the immediately preceding frame. This is justified becausestatistically video images change predominantly just slightly from frameto frame.

Preferably, the enhancer functions with an asynchronous regime, to allowthe device to be easily installed both into digital and analoguecameras.

The correcting transform of the video signal, performed by the enhancer,is preferably smooth and therefore reduces the effect of amplificationof visible noise, which is one of the drawbacks of standard histogramequalising technique (see William K. Pratt, “Digital Image Processing”,John Wiley and Sons, 1978, pp. 307-318).

Advantages of an analogue device based on the proposed technique includesmallness in size and significantly less power consumption in comparisonwith a digital implementation.

The use of analogue technology for nonlinear image processing allowsmaximizing of the dynamic range, since the nonlinear processing takesplace before the digitizing.

The described analogue systems are capable of real-time videoprocessing.

The suggested algorithm can be implemented as a monolithic analogueIntegrated Circuit (IC); such a chip could be installed both intodigital and into analogue video cameras. The size of an analogue ICimplementing the histogram equalization could be relatively small andthe power consumption relatively low in comparison with digital systems.Such an IC could be used e.g. in security cameras, where the cost andthe power consumption could be critical. Other applications are inhigh-speed cameras, where digital image processing systems might not befast enough, or in highest quality cameras, where the wide dynamic rangeof CCD (which could be higher than 2500:1) should be maximallypreserved. In this case the use of the analogue equalizers could bejustified, since it will allow avoiding the noise of digitizing, causedby the ADC.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic illustration showing a computational procedureused in the present invention.

FIG. 2 shows examples of a weight function F(I), which represents theasymmetry in perception of dark (small I) and bright (large I) by thehuman eye. The smaller the parameter Δ, the greater the degree ofasymmetry.

FIG. 3 shows examples of suitable orthonormal basis functions P_(i)(x)together with their antiderivatives Q_(i)(x); FIG. 3( a) shows theLegendre polynomial basis; FIG. 3( b) shows the piecewise basis.

FIG. 4 shows schematic illustrations of computational procedures forcolour image enhancement.

FIG. 5 is a schematic illustration of an analogue synthesiser ofLegendre polynomials P_(i)(x).

FIG. 6 shows schematically the generation of functions using an analoguecircuit. FIG. 6( b) shows an analogue circuit to generate Q_(i)(x),shown in FIG. 6( c), the antiderivatives of P_(i)(x), shown in FIG. 6(a).

FIG. 7 is a schematic illustration of a synthesiser of functions used ina piecewise transfer function version of an analogue hardwareimplementation of an image processor.

FIG. 8 is a schematic illustration of a 2D spatial low-pass filter.

FIG. 9 is a schematic illustration of a synthesiser of 2D orthonormalfunctions.

FIG. 10 is a schematic illustration of an analogue synthesiser oforthonormal 1D functions used in the synthesiser of FIG. 9.

FIG. 11 shows Cosine basis functions and Legendre polynomials.

FIG. 12 shows images comparing results of different imagetransformations; FIG. 12( a) is the initial image, 12(b) is the localhistogram equalized image, and 12(c) is the ORMIT-transformed image.

FIG. 13 is a schematic block diagram of a fast software implementationof the low-pass filter.

FIG. 14 illustrates the Sigmoid function.

FIG. 15 shows flow diagrams representing: (a) the procedure of thecombined ORMIT-JPEG compression, (b) and (c) the procedures ofdecompression. In the case (c) the new parameters of the ORMIT arepreset at the stage of decompression.

FIG. 16 schematically illustrates the dynamic range, and brightnessdistribution, at different stages of image processing.

FIG. 17 shows a schematic illustration of a polynomial histogramequalizer. The system is capable of real-time processing of analoguevideo signals.

FIG. 18 shows a schematic illustration of a piecewise histogramequalizer. The system is capable of real-time processing of analoguevideo signals.

FIG. 19 shows examples of “Hold” and “Reset” signals generated by theSync-selector, shown in FIG. 18.

DETAILED DESCRIPTION OF THE INVENTION

In various embodiments of the invention, data processing apparatusimplements a new image improvement transform, which is described by thefollowing equation:I _(out)=Σ_(i=0) ^(N)α_(i)(I)LPF _(Ω) _(i) [P _(i)(F(I))]·Q_(i)(F(I))+β(I)

where P_(i)(γ) is an orthogonal basis of functions of γ defined at therange 0<γ<1; and Q_(i)(..) are antiderivatives of P_(i)(..), i.e.Q_(i)(F(I))=∫₀ ^(F(I))P_(i)(η)dη, or some approximation thereto,LPF_(Ω)[..] is an operator of low-pass spatial filtering; Ω_(i) is acut-off frequency of the low-pass filter, F(..) is a weighting function,β(I) is preferably not equal to zero. Ω_(i) is preferably different fordifferent i such that the transformation has a different degree ofspatial nonuniformity in different brightness domains.

In one embodiment, the intensity value I(x,y) for each position (x,y) ofa greyscale image is adjusted to generate an intensity valueI_(out)(x,y) (improved image) for each position in accordance withI _(out)=αΣ_(i=0) ^(N) LPF _(Ω) [P _(i)(F(I))]·∫₀ ^(F(I)) P_(i)(η)dη+(1−α)I  (0.1)that is I_(out)=α·Σ_(i=0) ^(N)LPF_(Ω)[P_(i)(F(I))]·Q_(i)(F(I))+(1−α)I,where Q_(i)(F(I))=∫₀ ^(F(I))P_(i)(η)dη

Here α is the “strength” of transform.

The element α represents the proportion of mixture of the fullyequalised image (first term of the transform) with the initial image I.Thus the selection of appropriate strength of transform provides acompromise between the dynamic range compression and the natural look ofan image. In many cases, the contrast in dark regions of an image can beincreased without significant loss of the natural look of the overalltransformed image. Whilst a may be a constant, it may vary in accordancewith intensity as α(I).

The following expressions for α(r) and β(I) are used in one example:

${{\alpha(I)} = {\frac{1}{2} - {\frac{1}{2}{\tanh\left( {{4\frac{\log\left( {\frac{I}{\Delta} + 1} \right)}{\log\left( {\frac{1}{\Delta} + 1} \right)}} - 2} \right)}}}},{{{\beta(I)} = {I \cdot \left\lbrack {\frac{1}{2} + {\frac{1}{2}{\tanh\left( {{4\frac{\log\left( {\frac{I}{\Delta} + 1} \right)}{\log\left( {\frac{1}{\Delta} + 1} \right)}} - 2} \right)}}} \right\rbrack}};{or}}$${{\alpha(I)} = {A\left\lbrack {\frac{1}{2} - {\frac{1}{2}{\tanh\left( {{4\frac{\log\left( {\frac{I}{\Delta} + 1} \right)}{\log\left( {\frac{1}{\Delta} + 1} \right)}} - 2} \right)}}} \right\rbrack}};$$\begin{matrix}{{\beta(I)} = {I \cdot \left( {{\left( {1 - A} \right) \cdot \left\lbrack {\frac{1}{2} - {\frac{1}{2}\tanh\left( {{4\frac{\log\left( {\frac{I}{\Delta} + 1} \right)}{\log\left( {\frac{1}{\Delta} + 1} \right)}} - 2} \right)}} \right\rbrack} +} \right.}} \\\left. \left\lbrack {\frac{1}{2} + {\frac{1}{2}{\tanh\left( {{4\frac{\log\left( {\frac{I}{\Delta} + 1} \right)}{\log\left( {\frac{1}{\Delta} + 1} \right)}} - 2} \right)}}} \right\rbrack \right)\end{matrix}$

The functional forms of the strength functions α(I) and β(I) arepreferably chosen so that the transform is stronger (higher α) in thedarker parts of the image (low I) and becomes progressively weaker inbrighter parts. This allows stronger enhancement of shadows withoutaffecting the bright regions.

The weighting function F(I) is used in the algorithm to take intoaccount a property of the human eye approximated by a function which wewill refer to herein as a logarithm transfer function:

$\begin{matrix}{{{F(I)} = \frac{{\log\left( {I + \Delta} \right)} - {\log(\Delta)}}{{\log\left( {1 + \Delta} \right)} - {\log(\Delta)}}},} & (0.2)\end{matrix}$where 0<I<1, Δ is the parameter of “asymmetry” of “dark” and “light”,which provides stronger adaptive capability to the dark part of thebrightness domain of an image than to the bright one (thus the algorithmis more “active” in the dark regions of an image than in the brightones). The dark part may be defined to include those regions which areat less than half the maximum intensity, preferably less than onequarter. To provide a degree of dynamic range compression, we selectΔ<1. Reducing Δ produces a stronger dynamic range compression. Inpractice, a preferred range is 0.001<Δ<0.1. FIG. 2 illustrates thelogarithm transfer function for ranges in Δ from Δ=1 to Δ=0.001.

It should be noted that alternative functions, other than a logarithmicfunction, may also be used to approximate the properties of the humaneye. By the appropriate selection of a (the strength of transform),spatial nonuniformity of the transform (the “window” of the transform)represented by Ω, the weighting function F(..) and basic functionsP_(i)(..), it is possible to provide good quality image improvement,i.e. the dynamic range compression combined with the provision of“natural” looking transformed images.

FIG. 1 shows a schematic diagram of a computational procedure toevaluate equation (0.1) for each pixel of the input image given bycoordinates (x,y). Here F(I) is the asymmetric weight function definedby equation (0.2); P_(i) is the i^(th) member of a basis of Legendrepolynomials or piecewise functions (as described in the text); Q_(i) isthe antiderivative of P_(i); LPF is a two-dimensional low-pass filter asdescribed in the text. This could be coded in software directly by anexperienced programmer, or implemented in analogue hardware using knownelements. In the latter case, {circle around (×)} is a multiplierelement, Σ a summator; the nonlinear unit F(I) can be implemented usingstandard circuitry and the other elements are described elsewhereherein.

Different implementations of the ORMIT transform are suggested. A firstis based on orthogonal polynomials: P_(i)(γ). Legendre polynomials (seeFIG. 3( a)) in particular are selected in one embodiment. A secondimplementation is based on the piecewise functions (see FIG. 3( b)).Computer software implementations may be produced in an image processingapplication; analogue hardware implementations of these implementationsof the ORMIT transform are described below.

Analogue Hardware Implementations

Here we will consider possible analogue implementations of imageimprovement system, shown in the Figures.

Here x(t) refers to an image signal, and represents the intensitydependence during one frame of a video signal (0<t<T).

Both the “polynomial” and “piecewise” versions of analogueimplementation of the ORMIT transform may be based on an analogueimplementation of the low-pass filter, such as that shown in FIG. 8.FIG. 8 shows a schematic diagram of the 2D spatial low-pass filter (LPF)employing analogue circuitry. It takes as input the signal P_(i)(I),which could be a member of a Legendre or a piecewise basis. In thediagram, ∫ is an integrator element, SAMPLE/HOLD is a standardsample/hold amplifier and the synthesiser may be arranged as shown inFIG. 9. This filter employs the 2D basis of orthonormal functionsΠ_(ij.)(x,y). 2D orthonormal functions Π_(ij)(x,y) could be generated onthe basis of 1D orthogonal functions Π_(i)(x) and Π_(j)(y) (see FIG. 9).In FIG. 9, 1D Legendre polynomial or cosine functions (FIG. 11) Π_(i)are generated from input signals x,y. These are combined into their 2Dcounterparts Φ_(ij)(x,y) which are used in the LPF generator shown inFIG. 8. Synthesisers of 1D orthogonal functions Π_(i)(x) (adaptive andwith fixed coefficients) are shown in FIG. 5 and FIG. 10. The adaptivesynthesiser of orthogonal functions could generate:

1. Legendre polynomials if x(t) is the sawtooth waveform signal, thatis, the signal x(t)=SAW(t)=2REM(t/T)−1, where REM(..) is the remainderafter division, t is the time, T is the period of the sawtooth signal(more information about the analogue synthesiser of orthogonal functionscan be found in [1]);

2. Cosine basis of functions if x(t)=sin (πSAW(t)).

A “fixed” synthesiser of Legendre polynomials can be based on thefollowing recurrence relations (see K. B. Datta and B. M. “OrthogonalFunctions in Systems and Control”, Advanced Series in Electrical andComputer Engineering, Mohan publisher, World Scientific Pub Co, 1995):P(x)=1, P _(i)=(x)=x, P _(i)(x)=a _(i) P _(i−2)(x)+b _(i) xP_(i−1)(x),  (0.3)where

$a_{i} = {{\frac{i - 1}{i}\mspace{14mu}{and}\mspace{14mu} b_{i}} = {\frac{{2i} - 1}{i}.}}$

The scheme of such a synthesizer is shown in FIG. 5. The first fivecoefficients a_(i) and b_(i) are shown in the Table 1 below.

TABLE 1 $\begin{matrix}i & 1 & 2 & 3 & 4 & 5 \\a_{i} & 0 & {- \frac{1}{2}} & {- \frac{2}{3}} & {- \frac{3}{4}} & {- \frac{4}{5}} \\b_{i} & 1 & \frac{3}{2} & \frac{5}{3} & \frac{7}{4} & \frac{9}{5}\end{matrix}\quad$

The same recurrence relations (0.3) are valid for the Cosine basis offunctions. In this case the coefficients a_(i)=−1, b_(i)=2 for any i.

The Cosine basis could be used as an alternative to the Legendrepolynomials in the analogue low-pass filter. Both could be synthesisedefficiently in an analogue circuit using methods described in the text.The cosine basis has the advantage that the transform is more spatiallyuniform (free of “edge effects”).

Polynomial Version of the ORMIT Transform

This version of the ORMIT transform employs Legendre polynomials notonly for low-pass filtering, but also as the basic functions P_(i)(x)that used for computation of the transfer function of the ORMIT (seeFIG. 1 and FIG. 3( a)). Other basic functions that used for constructionof the ORMIT's transfer function are Q_(i)(x). The synthesizer of theQ_(i)(x) polynomials is shown in FIG. 6. Here, ⊕ is a differentialamplifier. Another version employs the Legendre polynomials for thebasic functions P_(i)(x) used in the transfer function of ORMIT, and thecosine functions for the low-pass filtering. A block-scheme of a fastsoftware implementation of the low-pass filter is shown in FIG. 13. Thealgorithm includes 1) image resizing, 2) a 2D Discrete Cosine Transform(DCT2), 3) an apodisation (Gauss) function, which zeroes the high orderspectral components of the resized image, 4) an inverse DCT (IDCT2),which gives a smooth approximation of the resized image, 5) the bilinearinterpolation procedure, which gives a smooth approximation of the inputimage (sizes: X,Y), that is the output of the low-pass filter.

Mixed Polynomial-Piecewise Version of the ORMIT Transform

This version of the ORMIT transform employs the basic functions P_(i)(x)and Q_(i)(x) shown in FIG. 3( b). A synthesiser producing thesefunctions is shown in FIG. 7. The functions produced by differentelements are illustrated on the right hand side of FIG. 7. In FIG. 7,the functions are non-overlapping, but alternatively two subsets ofmutually orthogonal, overlapping functions may also be used. In FIG. 7,the lower portions illustrate the signals output by the amplifierelements used in the arrangement illustrated in the main part of theFigure. The low-pass filter may either employ Legendre polynomialfunctions or cosine functions, as described above.

Further Details of Analogue Inplementations

To illustrate how an actual analogue hardware implementation would beconstructed in the Legendre polynomial case, consider the following.

The coefficients a_(i) and b_(i) could be stored into laser-trimmedresistors at the stage of manufacturing of an analogue ApplicationSpecific Integrated Circuit (ASIC). It should be noted here that theadaptive system of the adjustment of the a_(i) coefficients, which wasproposed in V. Chesnokov, “Analog Synthesizer of Orthogonal Signals”,IEEE Transactions on Circuits and Systems—II: Analog and Digital SignalProcessing, 47 (2000), No. 2, pp. 125-132 (and which is illustrated inFIG. 5), could be used in the manufacture of an analogue ASIC containingthe synthesizer of Legendre polynomials. This could be directlyimplemented as an analogue circuit using multipliers, summators and, forexample, trimmed resistors to supply the parameters a_(i), b_(i).

Let us consider now the procedure of the Q-polynomials synthesis. TheLegendre polynomials, defined by the recurrence relations (0.3), satisfythe following relations (see K. B. Datta and B. M. “Orthogonal Functionsin Systems and Control”, Advanced Series in Electrical and ComputerEngineering, Mohan Publisher, World Scientific Pub Co, 1995):

_(i+1)(x)−

_(i−1)(x)=(2i+1)·P _(i)(x).  (0.4)

Integrating Equation (0.4) in the range (−1, x) and taking intoconsideration the property of the Legendre polynomials (see Eq.(0.3)):P_(i)(1)=i and P_(i)(−1)=(−1)^(i) for any i (see FIG. 6 a), gives:

$\begin{matrix}{{Q_{i}(x)} = {\frac{1}{{2i} + 1}\left( {{P_{i + 1}(x)} - {P_{i - 1}(x)}} \right)}} & (0.5)\end{matrix}$

Here we have defined Q-polynomials as: Q_(i)(x)=∫⁻¹ ^(x)P_(i)(x)dx. Theprocedure of the Q-polynomials synthesis is illustrated in FIG. 6. Itshould be noted here that the Legendre polynomials defined by therecurrence relations (0.3) are not normalized. The normalized andnon-normalized Legendre polynomials respectively are shown in the FIG.3( a) (where the normalized Legendre polynomials and correspondingQ-polynomials are shown) and FIG. 6( a). Therefore to implement thescheme, shown in FIG. 17, the apparatus should perform the normalizationof the signals generated by synthesizers of the Legendre polynomials andQ-polynomials shown in the arrangement of FIG. 6. These normalisingcoefficients could be realized by the constants of integration ofintegrators shown in FIG. 17, so that different integrators should havedifferent constants of integration μ_(i) i.e.

F_(i) = μ_(i)∫₀^(τ)f_(i)(t) 𝕕t.

The task of the sync-selector is the extraction of the vertical syncpulses. The “hold” signal should be generated at the beginning of thevertical sync pulse, whereas the “reset” pulse should be generated justafter the “hold” pulse and before the start of the next frame (see FIG.19). This scheme (as well as the polynomial histogram equaliser)equalises the subsequent frame on the basis of information taken fromthe previous frame.

We also suggest a simplified implementation wherein the non-uniformityparameters of the ORMIT transform, the Ω_(i), are all set to zero. Thecorresponding circuitry in different embodiments is illustrated in FIGS.17 and 18. Note that, in FIG. 18, the lower portions illustrate thesignals output by the amplifier elements used in the arrangementillustrated in the main part of the Figure. The circuitry is simplifiedbecause it does not require the presence of a low-pass filter. Althoughthe improvement of images is not as effective as the full ORMITtransform, in many cases it may be sufficient to give a good enhancementof the image stream.

In one embodiment, an aim is to provide histogram equalization of acontinuous (both in time and in magnitude) video signal, which isspatially uniform. Legendre polynomials are used as a basis oforthogonal functions (although it is possible to use other orthogonalfunctions) since they could be implemented in analogue hardware (seeFIG. 5), which allows the purely analogue implementation of a histogramequalizer.

The general result is:

$\begin{matrix}{{y = {\sum\limits_{i = 0}^{\infty}\;{\left\langle {P_{i}(x)} \right\rangle{\int_{0}^{x}{{P_{i}(x)}\ {\mathbb{d}x}}}}}},} & (0.6)\end{matrix}$

where y is the intensity of the output signal, and the operator

..

is the arithmetic mean (or mathematical expectation) value for the wholepicture. In case of video-signal x(t) the arithmetic mean is

$\left\langle {P_{i}(x)} \right\rangle = {\frac{1}{T}{\int_{0}^{T}{{P_{i}(x)}\ {{\mathbb{d}t}.}}}}$

If we denote the antiderivative of P_(i)(x) as Q_(i)(x): Q_(i)(x)≡∫₀^(x) P_(i)(x)dx we have:

$y = {\sum\limits_{i = 0}^{\infty}\;{\left\langle {P_{i}(x)} \right\rangle{{Q_{i}(x)}.}}}$

If N→∞ the output video signal's y(t) brightness histogram will beuniform. In practice however N should be restricted, first: to providethe low computation intensity of the procedure, and second: to providesmooth histogram equalization. Computer modeling has showed thatpictures may be considerably improved even at N as small as 3. It shouldbe noted here that there is a probability of situations when thetransfer function y(x), which is approximating the cumulative histogramH(x), is not a monotonically increasing function. This gives animpression of an unnatural image and in most cases should be avoided.This could be achieved by introducing a coefficient of strength of thetransform—α.I _(out) =αI′+(1−α)I, where 0<α<1.

In effect, the output image is a mixture of the input image x and theequalized one y. Decreasing of α permits avoidance of the negativity ofthe derivative of the transfer function

$\frac{\partial x_{out}}{\partial x} < 0.$Normally it is sufficient to decrease α down to 0.5. The “strength” oftransform coefficient α can be adjusted manually by user of a videocamera (to get an appropriate image) or automatically by an adaptiveprocedure, which can be performed e.g. during the time between frames.

A straight implementation of the above-described algorithm assumes thatfor each frame the integrals

∫₀^(T)P_(i)(2x(t) − 1) 𝕕tshould be calculated to establish the transfer function for modificationof the current frame. This makes a straight analogue implementationdifficult unless the analogue signal's delay line, “remembering” thewhole frame is used. Nevertheless, the fact that the framesstatistically are just slightly changing from frame to frame, gives theopportunity of real-time analogue implementation of this algorithm. Inthis case the statistical information from a previous frame (representedby the above mentioned integrals) is used for modification of subsequentframe. On the scheme of the equalizer, which will be described insubsequent sections, (see FIG. 17) the integration and subsequentplacement of the result into a sample-hold amplifier are implementingthis procedure.

An important additional particular case is a basis of square functionsP_(i)(x), as are shown in the FIG. 3( b). Such basic functions are alsosuitable for analogue implementation of the histogram equaliser.

One of the typical disadvantages of the standard histogram equalizationtechnique is that it increases the visible noise of pictures (see e.g.William K. Pratt, “Digital Image Processing”, John Wiley and Sons, 1978,pp. 307-318) as well as makes images look unnatural. The noise (which isthe noise of digitizing plus the noise of CCD itself) amplification as aresult of a nonlinear image transform is a well known phenomenon. Thereason is that the cumulative histogram (therefore—the transferfunction) of typical images could have very large derivative (whichmeans nonuniformity of histogram).

Unlike the transfer function of standard histogram equalization, thetransfer function of the proposed device polynomial histogram equalizer)is very smooth, since it is composed from the low order polynomials(e.g. 3^(rd) order). It results in the fact that the processed imagesappear less noisy and more natural in comparison with ones processedwith a standard histogram equalization procedure.

The proposed device has the ability to improve a high percentage ofimages whereas the probability of worsening of images is very small.

In fact, such a property, as the ability of improvement of any kind ofpicture is not absolutely necessary (although desirable) for imageenhancement systems installed into video camera, since any effect (andin particular the suggested-smooth histogram equalization algorithm) canhave an adjustable strength (or can be switched off by the user).

Software Implementations

It should be noted that the processing of digital video images via asoftware implementation of ORMIT is performed in essentially the samemanner as described herein in relation to analogue hardwareimplementations, using functional software modules in place of thecircuitry described above and illustrated in the accompanying Figures.

Handling of Colour Images

Different schemes could be used for the processing of colour images. Thesimplest case is shown in FIG. 4( a). The element “ORMIT” performs theimage enhancement procedure. For any given pixel, R (R′) is the input(output) intensity in the red channel, and analogously for othercolours. I (I′) is the overall input (output) intensity, or equivalentlybrightness. {circle around (×)} denotes multiplication, Σ denotessummation. The W parameters are standard weighting factors. In the firstcase at the first stage the intensity signal I is produced for eachpixel of an image in accordance with formula I=W_(R)R+W_(G)G+W_(B)B (oralternatively the corresponding root mean square sum), where R, G and Bare values of red, green and blue.

The next stage is the ORMIT transform of the intensity signal, whichresults in the improved intensity signal I_(out). The final stage is themodification of each colour channel in accordance with the sameproportion as the modification of the intensity:

${R^{\prime} = {\frac{I_{out}}{I}R}},{G^{\prime} = {{\frac{I_{out}}{I}G\mspace{14mu}{and}\mspace{14mu} B^{\prime}} = {\frac{I_{out}}{I}{B.}}}}$Such a scheme improves the intensity distribution of an image withoutmodification of the hue and saturation.

Another scheme, employing the ORMIT is shown in FIG. 4( b). Such ascheme provides the spatial equalisation of different colour channels,so that not only the whole image has balanced colours, but also anyregion of an image (with the sizes larger than the size of “window”

${l_{\Omega} = \frac{1}{\Omega}},$where Ω is the cut-off frequency of the low-pass filter used in theORMIT transform) is colour-balanced. A wide range of colour correctionprocedures may employ the ORAM transform.Reversibility of the ORMIT Transform

Since the ORMIT transform is a smooth transform, it could be describedby a small number of parameters (typically the whole set of parametersis less than 1 kB, and preferably less than 100 bytes).

In many cases and in particular in the case of the piecewise version ofthe ORMIT transform, it is possible to restore the initial image on thebasis of the parameters of the transform and the ORMIT-transformedimage.

Let us consider the piecewise version of the ORMIT transform:I _(out)=α·Σ_(i=0) ^(N) LPF _(Ω) [P _(i)(F(I))]·Q _(i)(F(I))+(1−α)I,

where P_(i)(..) and Q_(i)(..) are shown in FIG. 3( b). The parametersdescribing the ORMIT transform are:

1. Δ, which defines the selected F(I),

2. Ω, the “parameter of nonuniformity”, which for the present embodimentwill be set as a constant for all i, and

3. the amplitudes of spectral components C_(kl) ^((i)) for differentbasic functions P_(i)(I(x,y)), which are calculated during the low-passfiltering and are the result of the 2D Cosine Transform and subsequentGauss-weighting function:

$C_{kl}^{(i)} = {{{DCT2}\left( {P_{1}\left( {F(I)} \right)} \right)}{\left\{ {k,l} \right\} \cdot {\exp\left( {{- \left( {\frac{k^{2}}{X^{2}} + \frac{l^{2}}{Y^{2}}} \right)}{\Omega^{2} \cdot {\min\left( {X^{2},Y^{2}} \right)}}} \right)}}}$(see FIG. 13), where X and Y are the dimensions of an image, Ω is theparameter of the spatial nonuniformity, which is the relation of the“window” size I_(Ω) (in any region of image with larger sizes than 1_(Ω)the histogram related to this region will be equalised) to the smallestof the dimensions of the image. The parameter of nonuniformity could be0<Ω<10, but in most cases is less than 3. Since the Gauss function isdecreasing very quickly with increasing k or 1, only a few parametersC_(kl) ^((i)) (with 0≦k,I<3÷10) will have nonzero values and thus willbe required to define the transform.

The piecewise ORMIT transformation therefore can be represented asI _(out)=α·Σ_(i=0) ^(N) IDCT2[C _(kl) ^((i))]·Q_(i)(F(I))+(1−α)I,orI _(out)=α·Σ_(i=0) ^(N) W _(i)(x,y)·Q _(i)(F(I))+(1−α)I, whereW_(i)(x,y)≡IDCT2[C _(kl) ^((I)) _(].)

The reversed transform for the piecewise version of the ORMIT is definedby the following equations:

${{F(I)} = {\sum\limits_{i = 1}^{N}\;{{Sigmoid}\left( \frac{I_{out} - I_{i}}{{\alpha\; W_{i}} + \frac{1 - \alpha}{N}} \right)}}},$

where the Sigmoid(x) function is shown in FIG. 14; I_(i)=0; I_(i) aredefined by

${I_{i + 1} = {{\sum\limits_{i}^{i}{\alpha \cdot W_{i}}} + {i\frac{1 - \alpha}{N}}}};$

Since the function F(I) is monotonically increasing, we can get theinverse expression I(F) taking into account the definition of F(I)(0.2). For the reversed piecewise ORMIT transform we thus have:I=exp(F·(log(1+Δ)−log(Δ))+log(Δ))−Δ.

The polynomial implementation of the reversible ORMIT transform (as wellas the most of other possible implementations with other basicfunctions) has no exact (analytical) representation, but a fastapproximating implementation of the reversed transform for polynomialversion of the ORMIT transform can also be produced.

An Enhancement of Lossy Compression Algorithms

This section describes the combination of a dynamic range compressionalgorithm, such as (but not limited to) the ORMIT transform, with alossy compression algorithm such as JPEG.

Lossy compression of images is widely used. This is because the storagesize of the compressed image can be as much as 10 to 100 times smallerthan the size of the original image, whereas the compressed and originalimages are virtually indistinguishable in appearance.

However, images which have undergone lossy compression are not suitablefor post-processing, such as is used for the analysis of medical images(X-rays, MRI etc) and in the manipulation of photographic images oncomputer. Post-processing of this kind amplifies the noise in thecompressed picture, resulting in unacceptable artifacts in the processedimage. For this reason, lossless formats (e.g. TIFF, BMP, PNG, losslessJPEG) are used. But these have the disadvantage that they result inlarge file sizes, which are costly to transmit and store.

A new method is here presented, which combines dynamic range compressionwith standard lossy image compression to produce an image which isapproximately the same file size as a standard lossy format, but whichretains the important information required for high-qualitypost-processing.

The procedure is shown in summary in FIGS. 15( a) and 15(b):

S1. Capture source image

S2. Apply dynamic range compression (e.g. ORMIT transform)

S3. Apply lossy compression (e.g. JPEG compression)

S4. Attach information describing step (S2) to header of JPEG image

S5. Transmit image

S6. Decompress JPEG image

S7. Decode header and reverse step (S2): dynamic range expansion(reverse ORMIT)

S8. Output image

An example illustrating the advantages of the ORMIT transform is nowdescribed. The size of the source image was 401 kB. Lossy compression ofthe source image resulted in an image having a file size of 22 kB.Post-processing of the compressed image, in this case brightnessamplification, led to the bright area in an originally dark of theimage. It was clear that the upper part of the image was well-preserved,whereas the lower part was of low quality.

Next, the source image was first transformed by the ORMIT transform.Compressing this image by JPEG at the same quality setting yielded animage having a file size of 28 kB—i.e. similar to the JPEG-onlycompressed file. To transmit this image, we added a 100 byte header filecomprising the parameters of the ORMIT transform. Subsequent JPEGdecompression and reversing the ORMIT transform using these parameters,led to the restored image. Post-processing of the restored image, inthis case by brightness amplification in the selected originally darkregion, provided a higher quality image in the dark region.

Comparing the resultant images, we concluded that the ORMITpre-processed image contains far more useful information in the darkregion than the simple JPEG image, whereas the file sizes of the twoimages are comparable.

Even using JPEG-only compression of the source image with a higher JPEGquality factor is less effective. The higher quality factor file had asize of 53 kB, almost twice as large as the ORMIT-enhanced image, andeven at this level of quality less information is retained in the darkregion (shown in brightness amplification in the lower box) than withORMIT pre-processing.

The reason for the result is that JPEG (or JPEG2000) uses a qualitycriterion which is based on the root mean square deviation between inputand compressed images, and does not discriminate between dark areas(where the human eye is most sensitive to contrast) and bright areas(where the eye is least sensitive). Application of dynamic rangecompression increases the efficiency of JPEG or other lossy compressionby preserving information in dark regions.

A further application is therefore the use of dynamic range compressionto decrease the file size of a JPEG or similar image, while retainingthe same visual appearance.

A further important application of strong dynamic range compression isas follows. Standard image formats use 8 bits per colour, whereas imagecapture devices can record e.g. 16 bits/colour. By applying dynamicrange compression, e.g. the ORMIT transform, followed by conversion to 8bits/colour, and then reversing the transformation at the point of imagereconstruction, allows e.g. 16 bit images to be transmitted using 8 bitimage formats. An example is the digital camera Agfa ePhoto1280. Thisproduces (after CCD and ADC) a 10 bit/colour image, from which only 8bit/colour are preserved by subsequent JPEG compression. Thus dynamicrange compression, and particularly the ORMIT transform, allow thecomplete use of the full potential of the high-precision hardware ofsuch cameras.

For the above procedure to be efficient, it is preferable that thealgorithm used for dynamic range compression can be described using asmall number of parameters (i.e., such a small number is required toreverse the transform). Because the ORMIT transform uses orthogonalfunctions, it satisfies this criterion.

Optimisation of Dynamic Range at Point of Image Reconstruction

A farther feature of the ORMIT transform is that, providing theparameters of the transformation have been appended to the image (seeFIG. 15( c)), the strength of the transform, and therefore the degree ofdynamic range compression, can be varied according to the capabilitiesof the display device. In the case of the ORMIT transform, this can bedone with minor additional computation (much less than that used toperform the initial transformation)

An example is the transmission of a JPEG-encoded image, pre-processedusing the ORMIT transform, and appended with the parameters of the ORMITtransform as in FIG. 15( a). These could be accommodated in the headerof the file according to the standard file specifications. There arevarious options for outputting this image, e.g. CRT screen, LCD screen,low-quality printer, high-quality printer. Each output device has adifferent effective dynamic range. A similar scheme may be used as inFIG. 15( b) except at the stage of reproduction, a further step S7A isadded wherein the parameters of the reverse ORMIT transform are variedat the output device to optimize the dynamic range of the image withthat of the output device. FIG. 16 shows the dynamic range, andbrightness distributions, at various stages of image processing—thedynamic range at the output device may be varied by use of a variableORMIT transform at the output device.

Another example is the transmission of an image without any lossycompression but again with ORMIT parameters appended to allowoptimisation of the image according to the characteristics of thedisplay/reproduction device. Another example is the application of theORMIT transform before lossy compression of digital video (e.g. MPEG).This would be especially useful if the movie is to be displayed on e.g.a LCD screen, which has a narrower dynamic range than a CRT screen. Afurther example is the conversion of movies shown at cinemas onto DVD orvideo, where the ORMIT transform can be used to compress from the (verywide) dynamic range of a cinema to the much narrower range of a CRT orLCD screen. The strength of transform can be controlled in real time atthe point of display.

Negative “Strength” of Transform

The ORMIT transform with negative “strength” of transform α<0 can beused as an efficient algorithm of noise suppression. Such an algorithmcould be used for example for real time retouching of images of humanfaces so that e.g. wrinkles will much less visible.

Applications and Advantages

Possible applications of the ORMIT transform include software fordigital still images (standalone, scanner's software, printer'ssoftware) and video improvement; digital still and video cameras;TV-sets and LCD displays; printers. Implementations in the analoguedomain include analogue video cameras; super-fast video cameras; highquality digital video cameras; security cameras (e.g. small, cheap andlow power consuming); X-ray and night-vision enhancement equipment.Other applications for the algorithm include hazy scene enhancement;multiple exposure based image synthesis for huge dynamic range scenes;advanced colour correction procedures; universal image improvement forTV-sets and displays.

Advantages of the ORMIT transform include:

1. It is a high quality image improvement algorithm (smooth in space andin brightness domain) combining efficient dynamic range compression withnatural look of the transformed images;

2. The digital implementation of the algorithm can be very fast (thefast implementation of the LPF based on bilinear interpolation and theDiscrete Cosine Transform is described above);

3. The algorithm can be implemented in purely analogue hardware.

Other features of the ORMIT transform (its reversibility and smallamount of parameters describing the transform, the case negative“strength” of transform, etc) are described above.

Advantages of the ORMIT over local histogram equalisation (adaptivehistogram equalisation) include:

1. High computational efficiency.

2. Smoothness in the space domain (the polynomial version of the ORMITtransform is smooth in the brightness domain too), whereas in many casesa rectangular window around each pixel is used for histogramequalisation in this window, thus such transforms cannot be consideredas spatially smooth. The non-smoothness of a transform could increasethe noise and artifacts of transformed images.

The asymmetry of the transform in terms of its adaptivity in “bright”and “dark” areas of image. The influence of the ORMIT transform in darkareas is much higher than in bright, which seems to be closer to thebehavior of the human eye, thus the transformed images look much morenatural than images enhanced by a local histogram equalization (adaptivehistogram equalization) procedure (see FIG. 12). FIG. 12 shows imagescomparing results of different image transformations; FIG. 12( a) is theinitial image, 12(b) is the local histogram equalized image, and 12(c)is the ORMIT-transformed image. The ORMIT is an algorithm adjusting itsmapping function (not just stretching the range of intensities as it is,for example, in the case of the Homomorphic Filtering of the Retinex™algorithm) adaptively to different parts of an image. Thus the ORMITtransform is a spatially nonuniform transform. This is a difference ofthis algorithm from the family of histogram-modifying algorithms, whichare described by a formula I′=Θ(I).

The ORMIT transform mapping function can be described by a followingformula: I′(x,y)=η(I(x,y), x, y), where

may be a continuous (or even smooth) function both in spatial (x, y) andin brightness I domains, that is:

${{{{{{{\frac{\partial\Theta}{\partial x}}_{y,I} < C_{x}},\frac{\partial\Theta}{\partial y}}}_{x,I} < {C_{y}\mspace{20mu}\frac{\partial\Theta}{\partial I}}}}_{x,y} < C_{I}},$where C_(x), C_(y) and C_(I) are some constants. This is a difference ofthe ORMIT transform from other local-histogram equalising algorithms,which usually are not smooth in the brightness and/or spatial domains.This results in appearance of artefacts in the transformed image.

Another difference is that the ORMIT transform is an orthogonaltransform, thus it may be embodied in a relatively computationallyefficient form, whereas low computational intensity is a known drawbackof pre-existing local histogram equalising algorithms.

One of the properties of the ORMIT transform is the adjustability ofstrength of transform, that is the output image is a mixture of amaximally equalised image with the input image, thus the compromisebetween the natural look of the transformed image and the dynamiccompression can be established by the user. Another difference of theORMIT from local histogram equalising algorithms is that ORMIT can havedifferent spatial nonuniformity for different domains of brightness of apicture. For example the transformation of bright parts of an imagecould be less spatially nonuniform than the transformation of darkparts. This provides for much higher flexibility of the algorithm andallows the creation of very high quality image improvement algorithms.

Another advantageous feature of the ORMIT is its reversibility.Furthermore, the ORMIT transformation may be entirely described by asmall amount of data (normally less that 100 Bytes).

Further Embodiments

The above embodiments are to be understood as illustrative examples ofthe invention only. Further embodiments of the invention are envisaged.It is to be understood that any feature described in relation to oneembodiment may also be used in other of the embodiments. Furthermore,equivalents and modifications not described above may also be employedwithout departing from the scope of the invention, which is defined inthe accompanying claims.

1. An image processing method comprising a transformation step ofprocessing an input image signal to generate an adjusted output imagesignal, wherein the intensity values I(x,y) for different positions(x,y) of an image are adjusted to generate an adjusted intensity valueI_(out)(x,y) in accordance with:I _(out)=Σ_(i=0) ^(N)α_(i)(I)LPF _(Ω) _(i) [P _(i)(F(I))]·Q_(i)(F(I))+β(I), where the P_(i)(γ) form an orthogonal basis offunctions of γ defined in a range 0<γ<1, the Q_(i)(..) areantiderivatives of the P_(i)(..): the Q_(i)(F(I))=∫₀ ^(F(i))=P_(i)(η)dηor approximations thereto, the LPF_(Ω)[..] is an operator of low-passspatial filtering; the Ω_(i) are cut-off frequencies of the low-passspatial filter, the F(..) is a weighting function, the α_(i)(l) arei^(th) members of a transformation step strength function, the β(I) is astrength function, the i and N are positive constants and the η is avariable.
 2. The image processing method according to claim 1, whereinthe β(I) is not equal to zero.
 3. The image processing method accordingto claim 1, wherein the Ω_(i) is different for different i such that thetransformation step has a different degree of spatial nonuniformity indifferent brightness domains.
 4. The image processing method accordingto claim 1, whereinI _(out)=α·Σ_(i=0) ^(N) LPF _(Ω) [P _(i)(F(I))]·Q _(i)(F(I))+(1+α)I,where the α is a transformation step strength value such that 0<α<1. 5.The image processing method according to claim 1, wherein saidorthogonal basis of functions are polynomial functions.
 6. The imageprocessing method according to claim 5, wherein said orthogonal basis offunctions are Legendre polynomial functions.
 7. The image processingmethod according to claim 4, wherein said orthogonal basis of functionsare piecewise linear mapping functions.
 8. The image processing methodaccording to claim 4, wherein said weighting function F(I) is selectedto provide a greater strength of transformation step in dark regionsthan in light regions.
 9. The image processing method according to claim8, wherein said weighting function varies substantially in accordancewith:${{F(I)} = \frac{{\log\left( {I + \Delta} \right)} - {\log(\Delta)}}{{\log\left( {1 + \Delta} \right)} - {\log(\Delta)}}},$where the Δ is a dark and light asymmetry parameter, and where Δ<1. 10.The method according to claim 1, wherein the transformation step isrepresentable by a transfer function which varies in shape nonlinearlyin a spatial domain.
 11. The method of claim 1, further comprising amethod of compressing an image signal, comprising conducting reversibledynamic range compression according to the method of claim 1, and lossydata compression during an image data compression procedure to generatecompressed image data.
 12. The method according to claim 11, comprisingconducting a reversible dynamic range compression algorithm followed bya lossy image data compression algorithm.
 13. The method according toclaim 11, comprising generating parameters relating to the dynamic rangecompression algorithm, and storing the generated parameters incombination with the compressed image data.
 14. The method of claim 1,further comprising a method of decompressing compressed image data,comprising conducting decompression, corresponding to a lossycompression, and a reverse of the dynamic range compression according tothe method of claim 1 in combination to provide a decompressed imagesignal.
 15. The decompressing method according to claim 14, comprisingreading parameters relating to said reversible dynamic range compressionalgorithm from data accompanying said compressed image data, and usingsaid parameters in said reverse dynamic range compression algorithm. 16.The decompressing method according to claim 14, comprising conductingsaid reverse dynamic range compression using one or more parametersselected in dependence on an image display method to be utilized.
 17. Amethod according to claim 11, wherein said dynamic range compressionalgorithm comprises processing an input image signal to generate anadjusted output image signal, wherein the intensity values I(x,y) fordifferent positions (x,y) of an image are adjusted to generate anadjusted intensity value I_(out)(x,y) in accordance with:I _(out)=Σ_(i=0) ^(N)α_(i)(I)LPF _(Ω) _(i) [P _(i)(F(I))]·Q_(i)(F(I))+β(I), where the P_(i)(γ) form an orthogonal basis offunctions of γ defined in a range 0<γ<1, the Q_(i)(..) areantiderivatives of the P_(i)(..): the Q_(i)(F(I))=∫₀ ^(F(I))P_(i)(η)dηor approximations thereto, the LPF_(Ω)[..] is an operator of low-passspatial filtering; the Ω_(i) are cut-off frequencies of the low-passspatial filter, the F(..) is a weighting function, the α_(i)(I) arei^(th) members of a transformation step strength function, the β(I) is astrength function, the i and N are positive constants and the η is avariable.
 18. A computer program adapted to implement the method ofclaim
 11. 19. An image processing method comprising the step ofprocessing an input image signal to generate an adjusted output imagesignal, wherein the intensity values (I) for different positions of animage are adjusted to generate an adjusted intensity value (I′) inaccordance with:$I^{\prime} = {\sum\limits_{i = 0}^{\infty}{\left\langle {P_{i}(I)} \right\rangle{\int_{0}^{I}{{P_{i}(\eta)}{\mathbb{d}\eta}}}}}$where P_(i)(..) defines a basis of orthogonal functions and the operator

..

means an arithmetic mean or mathematical expectation value for theimage, i is a positive constant and the η is a variable.
 20. The imageprocessing method according to claim 19, wherein the input image signalis a video signal represented by I(t) and the arithmetic mean is:$\left\langle {P_{i}(I)} \right\rangle = {\frac{1}{T}{\int_{0}^{T}{{P_{i}(I)}{{\mathbb{d}t}.}}}}$21. The image processing method according to claim 19, wherein saidorthogonal functions comprise a basis of square functions, orapproximations thereto.
 22. The image processing method according toclaim 19, wherein said orthogonal functions comprise Legendrepolynomials, or approximations thereto.
 23. An image processing devicecomprising a processor adapted to carry out the method of claim 1.